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Executive Summary, 



REGIONAL DISPARITIES IN KENTUCKY ACADEMIC INDEX SCORES 

Recent newspaper articles by columnist Bill Bishop in the Lexington Herald-Leader point 
to persistent regional disparities in achievement in Kentucky public schools. In spite of a mandate 
to create educational equality, the Kentucky Education Reform Act (KERA) has not eliminated 
inequalities in school performance scores and the problem may even be getting worse according 
to Mr. Bishop. The objective of the present study is to determine what disparities exist among 
Kentucky’s eight Service Center Regions. The analysis uses school Academic Index scores, from 
1 992-93 to 1 996-97, averaged by region. The regional averages (or means) are compared 
descriptively along with the statistical application of pairwise multiple comparisons. 

The study’s findings show that regional disparities in Academic Index scores clearly exist. 
These disparities are probably owing, at least in part, to contextual effects such as rural- 
metropolitan differences and socioeconomic influences. A goal of KERA should be to narrow the 
gap between the schools in different regions. To assess the accomplishment of this goal, it will be 
useful to regularly monitor disparities in the gains that regions achieve in their scores over time. 
The present analysis concludes that after five years, schools in the eight Service Regions have 
generally kept pace with one another. Thus, although greater equality is still an unrealized ideal 
under KERA, improvement has taken place with little evidence thus far of a “widening gap” 
between the higher- and lower-scoring regions. Another five years of data should tell if a 
widening gap between the regions has become a persistent problem. 



Introduction 



Recently, Lexington Herald-Leader columnist Bill Bishop pointed to persistent regional 
disparities in Kentucky schools, despite the State’s comprehensive program of educational reform. 
In a February 15, 1998 column entitled “Equality Was Heart of Reform,” Bishop (1998a) 
reported that the schools in Service Region 8 (Pikeville/Hazard/Prestonsburg, etc.) score lower 
than do schools in the other regions. Furthermore, Bishop noted that Region 8 schools are 
“falling behind the rest of the state.” 1 A March 1, 1998 column by Mr. Bishop (1998b) charges 
that due to 60 years of neglect “Eastern Kentucky has an accumulated education deficit.” 2 It is 
this deficit that accounts for the area’s lack of economic and social development today. 3 

Mr. Bishop has put his finger on an important issue. The Kentucky Education Reform Act 
of 1 990 (KERA) was founded on the ideal that all children of the Commonwealth should have 
equal educational opportunities. KERA is now in its seventh year, and five years of school 
performance data have been accumulated. It is not too early to begin assessing if the ideal of 
equality is being realized. 

The present study examines regional achievement in school Academic Index scores for the 
five-year period, from 1992-93 to 1996-97. The analysis makes a distinction between: (1) 
differences in scores that show which regions on average have scored higher and which have 

'Bishop (1998c) has repeated these views in an April 26, 1998 column. 

2 The area referred to in this article includes Service Region 7 (Ashland/Maysville/ 
Morehead, etc.). Region 8 (Pikeville/Hazard/Prestonsburg, etc.), and part of Region 6 
(Richmond/London/Williamsburg, etc.). 

3 Greenberg and Teixeira (1995), Brint (1998), and others have disputed the claim that 
there is a simple, direct relationship between the educational status of the population and 
economic development. 
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scored lower, and (2) changes in a region’s scores over time compared with the changes in other 
regions. Thus, the objective of the study is to find out what disparities in educational performance 
exist among Kentucky’s eight Service Center Regions (FIGURE A). 



Methodology 

Using Academic Index scores from 1 992-93 to 1 996-97, 4 an annual average score of the 
schools in each region was obtained to document regional trends. Next, a 5-year mean was 
calculated for each of the eight regions. Then, differences between the regions were analyzed 
descriptively and with a pairwise multiple comparison of the means. A further analysis identified 
regions that, have achieved statistically-determined greater or lesser gains in their average school 
scores. A problem arose at this juncture. The graph in FIGURE B (which is derived from 
TABLE 1 below) illustrates the nature of the problem. The graph traces the overall Kentucky 
average test scores by year, from 1992-93 to 1996-97, and also displays the annual scores of two 
below-average regions. Region 3 (Louisville/Jefferson Co., etc.) and Region 8 (Pikeville/Hazard/ 
Prestonsburg, etc.). For visual comparison the graph shows the trend in scores of the highest 
performing region in the State, Region 5 (Lexington/Frankfort/Winchester, etc.). As FIGURE B 



4 The 1 992-93 to 1 995-96 data were obtained from the corrected Cycle 2 results. The 
1996-97 (Cycle 3) data were then merged with these previously available data by matching school 
names. The merging of these two data sets posed a problem since the 1 996-97 data set includes 
scores not recorded in the previous data set as well as joint school scores. Joint schools are one 
or more schools that have been combined with another school for accountability purposes because 
the school does not contain both a 4th and 5th grade or both a 7th and 8th grade. Joint schools 
are the result of the expansion of testing at grades 5 and 7, which began in 1996-97. Each joint 
school has a combined record and an individual school record for the schools that comprise the 
joint school. In the present study, care was taken to expunge the joint school records. Only the- 
scores of individual schools were used. A missing values analysis found very slight differences in 
the means and standard deviations of the listwise values used in this study compared with the 
means and standard deviations of all values available for each year. 

2 




6 



Figure A 
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School Year 



illustrates, the statewide average score increased during the first three years, declined in the fourth 
year, and increased again in the fifth year. The present study will not attempt to explain why this 
kink in the trend occurred. 

The trend of Region 5 (Lexington/Frankfort/Winchester, etc.) has a shape similar to the 
Kentucky trend, although the Region 5 mean score is 2-3 points higher than the State average in 
each year. In contrast with this, the trend of Region 8 (Pikeville/Hazard/Prestonsburg, etc.) lies 
beneath the Kentucky trend. Region 8 mean scores are 2-3 points less than the State averages 
until the final year, 1996-97, when the trend flattens and the mean score of this region drops to 5 
points below the State average. Region 3 is near the State average for the first three years then 
drops below Region 8 in 1995-96. before recovering in the fifth year. How should changes in the 
scores be interpreted given these trends? The simple approach would be to compare the 
difference in school scores between 1992-93 and 1996-97, averaged for each of the eight 
regions. 5 But, when this procedure is used. Region 8 registers a negligible increase from 1995-96 
to 1996-97. As a result it has a significantly lower 5-year gain when compared with most of the 
other regions (see below, TABLE 4, upper figures). This result, however, is due entirely to the 
comparatively slight gain in 1996-97. If the difference were calculated between 1992-93 and 
1995-96, for example. Region 8 would have registered greater improvement than Region 3, and 
the gain in Region 8 would not deviate significantly from the gains achieved by the other regions. 6 

The lesson here is that the comparison of region-to-region gains in scores when based on 

5 This is the procedure that Bishop (1998c) follows in his April 26, 1998 column. His 
analysis focuses on mathematics scores, whereas the analysis in the present study uses the total 
Academic Index score. 

6 This analysis is not presented here but will be provided on request. 
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the differences between single-year scores can produce misleading results. To demonstrate this 
more comprehensively, two approaches to analyzing changes in the scores were used in this study. 
First, the simple approach just described which compares the difference between the first- and 
fifth-year scores was used. Since this approach does not control the volatility of single-year 
scores, it can exaggerate disparities (and similarities) between the regions. The second approach 
was to subtract the average of the first two years from the average of the fourth and the fifth years 
to obtain a measure of change. In other words, for each region, the 1992-93 and 1993-94 scores 
were averaged and subtracted from the average of the 1995-96 and 1996-97 scores. Compared 
to the first approach, this yields less volatile estimates of regional gains by smoothing out 
fluctuations in the differences between the first- and the last-year scores. 

Findings 

TABLE 1 displays the 5-year trends in the average Academic Index scores for the eight 
Service Center Regions. The trend for the entire State went as follows: In 1992-93, the score 
was 24.3. By 1994-95, it had increased to 36.2. Then in the following year, 1995-96, the score 
declined to 34.7, before increasing to 39.0 in 1996-97. Across the 5-year period, the scores for 
the entire State gained 14.7 points. The trends in TABLE 1 also reveal that Region 5 (Lexington/ 
Frankfort/Winchester, etc.) is consistently the highest-scoring region and Region 8 (Pikeville/ 
Hazard/Prestonsburg, etc.) is consistently the lowest-scoring region. Region 7 (Ashland/ 
Maysville/Morehead, etc.) registered scores that most closely approximate the entire State 
averages. A clearer picture of the differences between the regions emerges by averaging these 
trends. 
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TABLE 1 



REGIONAL AVERAGES OF SCHOOL ACADEMIC INDEX 
SCORES FROM 1992-93 TO 1996-97 



School Years 

Regions of 
the State 


1992-93 


1993-94 


1994-95 


1995-96 


1996-97 


Region #1 

(Paducah/Henderson/Murray, etc.) 


24.6 


31.1 


37.4 


34.8 


40.7 


Region #2 

(Bowling Green/Owensboro/ 
Elizabethtown, etc.) 


25.0 


31.9 


36.7 


36.0 


41.8 


Region #3 

(Louisville/Jefferson Co., etc.) 


23.3 


29.6 


35.9 


31.6 


36.7 


Region #4 

(Covington/Newport/Dayton, etc.) 


25.8 


34.3 


38.0 


36.4 


41.6 


Region #5 

(Lexington/Frankfort/Winchester, 

etc.) 


27.2 


33.8 


38.7 


37.3 


42.2 


Region #6 

(Richmond/London/Williamsburg, 

etc.) 


22.8 


29.7 


34.4 


33.3 


36.4 


Region #7 

( Ashland/Maysville/Morehead. etc.) 


24.9 


30.6 


35.9 


34.7 


39.6 


Region #8 

(Pikeville/Hazard/Prestonsburg, etc.) 


21.5 


28.7 


33.4 


32.8 


33.8 


Entire State 


24.3 


31.2 


36.2 


34.7 


39.0 
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TABLE 2 displays the means and standard deviations of the school scores after they were 
averaged for the 5-year period and broken down by Service Region. The data also reflect the 
lowest and highest scores posted for schools in each region. Region 5 (Lexington/Frankfort/ 
Winchester, etc.) has the highest mean score (M = 35.9), while Region 8 (Pikeville/Hazard/ 
Prestonsburg, etc.) has the lowest mean score (M = 30.0). Region 1 (Paducah/Henderson/ 
Murray, etc.). Region 2 (Bowling Green/Owensboro/Elizabethtown, etc.). Region 4 (Covington, 
Newport, Dayton, etc.). Region 5 (Lexington/Frankfort/ Winchester, etc.), and Region 7 
(Ashland/Maysville/Morehead, etc.) have mean scores that equal or exceed the State average. 
Region 3 (Louisville/Jefferson Co., etc.). Region 6 (Richmond/London/Williamsburg, etc.), and 
Region 8 (Pikeville/Hazard/Prestonsburg, etc.) fall below the State average. The school with the 
lowest average level of performance for the 5 -year period (M = 15.8) was in Region 3 (Louisville/ 
Jefferson Co., etc.). Region 5 had the school with the highest average level of performance 
(M = 70.5) during the same span of years. 

The descriptive data in TABLE 2 clearly suggest regional disparities, but they do not tell 
which regions had significantly lower or higher levels of school performance than another. A 
pairwise multiple comparison of the regional means is needed to find this out. 7 Looking at the 
negative differences (TABLE 3), Region 3 scores significantly lower than either Region 4 or 
Region 5 (the mean differences are -3.8 and -4.4 respectively). Region 6 scores lower than four 
other regions-Regions 1, 2, 4, and 5, while Region 8 scores lower than these same four regions 
and also lower than Region 7. Clearly, by this analysis. Region 6 and Region 8 on average have 



7 Since the variances were unequal, this analysis employed the statistical application of 
Tamhane’s T2 method of testing the pairwise multiple comparison of the group means. 
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TABLE 2 



SCHOOL ACADEMIC INDEX SCORES AVERAGED BY REGION 

FROM 1992-93 TO 1996-97 



Descriptive 


Number 


Regional 


: •!: • • 


Within-Region Range 


Measures 

Regions of v. 

the State 


of 

Schools 

(N) 


Mean 

Score 

(M) 


Standard 

Deviation 

(SD) 


Lowest 

School 

Score 


Highest 

School 

Score 


Region #1 

(Paducah/Henderson/Murray, etc.) 


135 


33.7 


5.6 


20.1 


47.2 


Region #2 

(Bowling Green/Owensboro/ 
Elizabethtown, etc.) 


154 


34.3 


5.8 


18.2 


52.7 


Region #3 

(Louisville/JefFerson Co., etc.) 


108 


31.4 


8.4 


15.8 


54.8 


Region #4 

(Covington/Newport/Dayton, etc.) 


137 


35.2 


7.3 


16.0 


60.4 


Region #5 

(Lexington/Frankfort/Winchester, etc.) 


154 


35.9 


7.9 


18.1 


70.5 


Region #6 

(Richmond/London/ Williamsburg, etc.) 


179 


31.3 


6.4 


19.9 


52.8 


Region #7 

(Ashland/Maysville/Morehead, etc.) 


100 


33.1 


6.7 


20.6 


58.2 


Region #8 

(Pikeville/Hazard/Prestonsburg. etc.) 


163 


30.0 


5.7 


19.0 • 


53.0 


(ENTIRE STATE) 


1,130 


33.1 


7.0 


15.8 


70.5 



9 




BEST COPY AVAILABLE 



15 



TABLE 3 



COMPARISONS OF AVERAGE REGIONAL DIFFERENCES IN SCHOOL 
ACADEMIC INDEX SCORES FROM 1992-93 TO 1996-97 



Regions of the State 


Region 

#1 : 


Region 

#2 


Region 

I #3 - : i 


Region 

J; f #4; 


Region 

#5 


Region 

#6 


Region 

#7 


Region 

#8 


Region #1 

(Paducah/Henderson/Murray, etc.) 


— 


— 


— 


— 


— 


2.4 


— 


3.7 


Region #2 

(Bowling Green/Owensboro/ 
Elizabethtown, etc.) 


— 


— 


— 


— 


— 


3.0 


— 


4.2 


Region #3 

(Louisville/Jefferson Co., etc.) 


— 


— 


... 


(-3.8) 


(-4.4) 


... 


— 


— 


Region #4 

(Covington/Newport/Dayton, etc.) 


— 


— 


3.8 


— 


— 


3.9 


— 


5.2 


Region #5 

(Lexington/Frankfort/Winchester, 

etc.) 


— 


— 


4.4 


— 


— 


4.6 


— 


5.8 


Region #6 

(Richmond/London/Williamsburg, 

etc.) 


(-2.4) 


(-3.0) 


— 


(-3.9) 


(-4.6) 


— 


— 


— 


Region #7 

(Ashland/Maysville/Morehead. etc.) 




— 


— 


— 


— 


— 


— 


3.1 


Region #8 

(Pikeville/Hazard/Prestonsburg, etc.) 


(-3.7) 


(-4.2) 


— 


(-5.2) 


(-5-8) 


— 


(-3-1) 


— 



NOTE : For emphasis negative figures are enclosed by parentheses. Only the mean differences that are 
significant at the 0.05 level are displayed 
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lower scores than a majority of the other regions. According to the multiple comparison analysis, 
the mean scores of Regions 1, 2, 4, 5, and 7 are not surpassed by any other region; and the 
differences between these regions are not significant. 

This study will not attempt to investigate why these regional disparities exist, but a partial 
explanation can be suggested for fixture research. The relative advantage of Region 4 (Covington/ 
Newport/Dayton, etc.) and Region 5 (Lexington/Frankfort/Winchester, etc.) could be owing to 
the high proportion of metropolitan (especially suburban) schools. Region 3 (Louisville/Jefferson 
Co., etc.) is totally metropolitan but many of its schools are also inner-city. This difference may 
be sufficient to account for the lower average score in Region 3 compared with Region 4 and 
Region 5. The lower performance of schools in Region 6 (Richmond/London/Williamsburg, etc.) 
and Region 8 (Pikeville/Hazard/Prestonsburg, etc.) require an alternative explanation. These 
regions have the highest proportion of rural schools in the State. These are also the regions with 
the lowest high school graduation rates and the lowest median household incomes (Reeves, 

1998). 

The above comparisons used the school Academic Index scores averaged by region, from 
1992-93 to 1996-97. A different way to compare the regions over the same years is to look at 
changes in scores that indicate differences in the mean or average amount of gain. This is 
especially pertinent for determining if a “widening gap” between the regions is actually coming 
about. As stated previously, two approaches were used to address this issue. The first approach 
was to calculate the simple mean difference in scores between the first and the fifth year. The 
second approach calculated the difference between the average score of the first two years and the 
average score of the last two years. Using these two approaches and the multiple comparison 



analysis produced strikingly different results. 8 

Following the first approach (TABLE 4, upper figures). Region 3 and Region 6 were 
found to have fallen behind Region 2. But it is Region 8 that has fared least well according to the 
findings of the first approach. The gains of this region lag behind the gains of Region 1, Region 2, 
Region 4, and Region 5. On the other hand, significant differences were not found between the 
gains of Region 1, Region 2, Region 4, Region 5, and Region 7. These results support Mr. 
Bishop’s argument (1998a, 1998c) that regional disparities exist and Region 8 in particular is 
falling behind other regions. A different conclusion emerges when the second approach to 
calculating the 5-year gain is used, however. 

Using the second approach (TABLE 4, lower figures). Region 3 was discovered to have 
experienced a shortfall in improving its scores compared with Region 1 and Region 2. The scores 
of the remaining pairs of regions were not sufficiently different to attain statistical significance. 

As explained above, the authors believe that this second approach to calculating the change in the 
scores moderates the exaggerations to which the first approach is subject and offers more reliable 
results. According to the second approach, the “mountain schools” of Region 8 are not falling 
behind the rest of the State, as Mr. Bishop has contended. Nevertheless, Bishop is right to be 
concerned about the possibility of increasing disparity between the regions. This concern is a 
good reason to continue regular monitoring of the regional Academic Index scores. 

Discussion 

Performance-based educational reform— holding schools accountable for what students 



8 As before, the statistical test for pairwise multiple comparison was Tamhane’s T2. 



TABLE 4 



COMPARISONS OF AVERAGE REGIONAL DIFFERENCES IN THE CHANGE IN 
SCHOOL ACADEMIC INDEX SCORES FROM 1992-93 TO 1996-97 




NOTE: For each comparison above, the upper figure is the simple mean difference in scores, from 1992-93 to 
1 996-97. The lower figure is a mean difference obtained by subtracting the average of the first two 
years from the average of the last two years. For emphasis negative figures are enclosed by 
parentheses^ Only the mean differences that are significant at the 0.05 level are displayed. 
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leam--is a nationwide issue (Ladd, 1996; Nelson, 1998). Kentucky has achieved national 
prominence because of its highly innovative and comprehensive reform program (Elmore, 
Abelmann, & Fuhrman, 1996; Guskey, 1994). The Kentucky Supreme Court decision issued in 
June of 1989, which led to the Kentucky Educational Reform Act of 1990, called for “an efficient 
system of common schools” (Miller, Noland, & Schaaf, 1990: ix, 2). Among other things, the 
characteristics of such an educational system would include: schools that are “substantially 
uniform throughout the State” and the provision of “equal educational opportunities to all 
Kentucky children.” Monitoring regional differences in the performance of schools is a useful 
way to assess if these goals are being met. It can help point out if the schools are providing an 
education that is “substantially uniform throughout the State.” Moreover, it shows indirectly the 
degree to which “equal educational opportunity” is being realized across the several regions. 

A consequence of this inquiry into regional disparities is that it forces one to be more 
precise about what uniformity and equal educational opportunity really mean. If these terms mean 
that test scores should be substantially uniform across the State and between the regions, then 
according to the results of the present study KERA has not succeeded. The comparisons of 
regional mean scores (TABLE 2 and TABLE 3) clearly show that significant differences between 
the regions exist. It is quite likely that these differences reflect contextual influences on 
education, including the historical circumstances to which Mr. Bishop refers (Bishop, 1998a; 
Bishop, 1998b; also Reeves, 1998). 

But if by “substantial uniformity” and “equal opportunity” one implies instead that 
constraints on the ability of schools to improve children’s performance should be kept to a 
minimum, the results obtained above (especially in TABLE 4, lower figures) are less pessimistic. 
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The evidence is that all regions are improving and most are keeping pace with one another. It 
would be preferable if lower scoring regions, such as Region 6 (Richmond/London/Williamsburg, 
etc.) and Region 8 (Pikeville/Hazard/Prestonsburg, etc.), could eventually gain on higher scoring 
regions, such as Region 4 (Covington/Newport/Dayton, etc.) and Region 5 (Lexington/Frankfort/ 
Winchester, etc.). This is a goal that should be pursued diligently by the State. , 

Based on five years of data, greater educational equality remains an unrealized ideal under 
KERA. Still, gains have been made in all of the Service Center Regions with little evidence thus 
far of a widening gap between the higher- and lower-scoring regions. Another five years of data 
should tell if a widening gap between the regions has become a persistent problem. 



Conclusions 



Based on the stated assumptions, identified limitations, and the displayed and discussed 
findings, an analysis of the data in response to the issues raised permits the following conclusions 
to be drawn: 

• Based on the analysis of 5 years of Academic Index scores, regional disparities persist 
under KERA. Schools in some regions of the State have higher performing students 
than do schools in other regions. 

• Lower scores are found in regions where there are a large number of inner-city 
schools, and where there are numerous rural schools and populations of a low 
socioeconomic status. 

• Most regions had similar gains or improvement in their test scores over the five-year 
period. Therefore, evidence that would confirm a “widening gap” between the regions 
is limited at this point in time. 
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